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Abstract 


A safe environment is a prerequisite for productive learning. This paper represents the 
first large-scale analysis of how feelings of safety at school affect educational outcomes. Using 
a unique longitudinal dataset of survey responses from New York City middle school students, 
the paper provides insight into the causal relationship between feelings of safety and academic 
achievement. The survey data include the reported feelings of safety for more than 340,000 
students annually from 2007-2010 in over 700 middle schools. Findings show consistent 
negative effects of feeling unsafe on test scores. The paper explores the mechanisms through 
which feeling unsafe in the classroom may impact test scores, and multiple robustness checks 
support the validity of the causal claim. 

Key Words: safety and fear; school environment; academic achievement; middle school 


Introduction 


Persistent racial and ethnic gaps in educational achievement have focused policy 
attention on school climate and safety as important elements of educational performance. In a 
special issue of Educational Researcher focused on safety and order in schools, Cornell and 
Mayer (2010) argue that school safety and school order are fundamental to studies of the 
achievement gap, teacher attrition, and student engagement. This paper represents the first 
large-scale analysis of how feelings of safety at school affect educational outcomes. If minority 
students feel differentially unsafe at school, school safety may be a primary factor contributing 
to racial and ethnic achievement gaps. 

Academic achievement is a critical step toward future success in adult life, increasing 
employment and earnings and the probability of other stabilizing life events such as marriage. 
Studies show that early childhood test scores are positively correlated with future labor market 
outcomes (Currie and Thomas 1999). Yet, black and Hispanic students consistently 
underperform on standardized tests compared to white and Asian students (U.S. Department of 
Education 2004, 2008). These racial gaps persist even as test scores have risen for all 
students. Gaps in achievement extend to college enrollment and completion rates and as a 
result, white students are more than twice as likely to earn a bachelor’s degree as black 
students (Western 2006). Educational gaps translate into differences in wealth accumulation 
over the life-course, differing rates of marriage (Schneider 2011), and disparities in future health 
outcomes (Freudenberg and Ruglis 2007; Fiscella and Kitzman 2009). Lower educational 
attainment is associated with an increased probability of arrest and incarceration: the risk of 
imprisonment is five times greater for black men with no college degree compared to white men 
with the same level of education (Lochner and Moretti 2004; Western 2006). Identifying the 
factors that contribute to these gaps is critical to narrowing disparities in later-life outcomes. 
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A safe environment is a prerequisite for productive learning (Maslow 1970; Piaget 1936). 
Feeling unsafe in the classroom may decrease concentration in class and performance on 
assessments, yet only a few studies have focused on the contribution of feelings of safety to 
educational achievement. Henrich and colleagues (2004) surveyed a sample of New Haven 
middle schools students and found that exposure to community violence is related to academic 
achievement and feelings of safety. However, the authors do not discuss the relationship 
between feeling unsafe and academic outcomes, and are unable to control for unobserved 
individual characteristics that might explain both feelings of safety and achievement. In his work 
on school discipline, Arum (2003) determines that feelings of safety are positively related to both 
behavioral and academic outcomes. He finds variation in the relationship between safety and 
academic outcomes by gender, with feelings of safety having larger positive association with 
test scores for females than males, and larger positive association with behavior (i.e. decreases 
in fighting) for males than females. 

If students feel unsafe at school, one response may be to stay home. Therefore, 
increased school absences may be the primary path through which feeling unsafe affects 
academic outcomes. Although many studies assert that missing school affects students 
negatively, few studies have empirically investigated the impact of being absent on academic 
achievement. A notable contribution to the literature is Gottfried’s (2010) work identifying the 
relationship between attendance and achievement in elementary and middle schools in 
Philadelphia. Using detailed student-level data and a school and classroom fixed effects 
approach, Gottfried finds that attendance and achievement are positively related. 

A related body of research investigates the relationship between school and 
neighborhood crime and disorder that affect achievement. Exposure to neighborhood violence 
affects students’ academic performance and that of their peers (Aizer 2008; Carrell and 
Hoekstra 2010; Delaney-Black et al. 2002; Gibson, Morris, and Beaver 2009; Ripski and 
Gregory 2009; Sharkey 2010). Studies relying on student or principal reports find that violence 
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within schools reduces school attendance, increases misbehavior, and reduces the likelihood of 
high school graduation and college attendance (Bowen and Bowen 1999; Chen 2007; Grogger 
1997). Peer disorder, such as bullying, is negatively related to achievement and is also 
associated with more serious school violence (Arseneault et al. 2006; Buhs, Ladd, and Herald 
2006; Glew et al. 2005; Schwartz, Gorman, Nakamoto and Toblin 2005; Skiba et al. 2004). In 
fact, these less serious incivilities are stronger predictors of feelings of safety than violent crimes 
or personal experiences of crime (Mayer 2010; Skiba, Simmons, Peterson, and Ford 2006). 

Research about racial and ethnic test score gaps identifies school and neighborhood 
contexts as sources of differences in test scores, but few studies have been able to adequately 
measure these contextual factors (Fryer and Levitt 2004). Research in progress provides 
evidence that black and Hispanic students are more likely to report feeling unsafe at school than 
white or Asian peers who attend the same schools and share the same classrooms (Author 
2012). Factors related to these racial gaps in safety include disciplinary fairness, school 
disorder, and racial tension. Research has also documented racial disparities in the use of 
school discipline, with higher rates of office referral, suspension, and expulsion for black 
students (Skiba et al. 2002), and in perceptions of disciplinary fairness (Arum 2003; Kupchik 
and Ellis 2008). Racial gaps in discipline may be directly linked to achievement gaps, because 
suspensions decrease academic performance for black middle and high schools students and 
contribute to late graduation and dropout (Davis and Jordan 1994; Raffaele Mendez 2003). 
Differences in disciplinary environments across schools may contribute to racial gaps in 
achievement, with black high school students achieving higher test scores in schools with more 
discipline (Arum and Velez 2010). Although there is a growing body of evidence about how 
exposure to violence and disorder affects students, and about racial and ethnic disparities in 
school discipline, we know less about the mechanisms that connect these phenomena to 
academic achievement. Feeling unsafe at school is one way that these factors may affect 
student performance. 
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This paper improves upon the previous literature in two important ways. First, I exploit a 
large, longitudinal data set of information about students, including their feelings of safety and 
perceptions of violence and disorder at school, linked to administrative academic records. The 
dataset represents a sample size that is many times larger than previous studies. Second, I 
provide insight into the causal relationship between feeling unsafe and academic performance 
through a variety of econometric methods and multiple robustness checks. 

PURP OSE AND HYPOTHESES 

The central question addressed in this paper is: Does feeling unsafe in the classroom 
affect student academic performance? Based on Maslow’s hierarchy of needs which places 
safety above only breathing, food, and water (Maslow 1970) and evidence that neighborhood 
conditions such as community violence affect children’s cognitive ability (i.e. Sharkey 2010), I 
hypothesize that feeling unsafe in the classroom will negatively affect student performance on 
standardized assessments. There may be both a direct effect of feeling unsafe on test scores, 
and an indirect effect (see Figure 1). Feeling unsafe in the classroom may directly affect test 
scores if it inhibits learning or distracts students as they take exams. Flowever, feeling unsafe 
may also affect academic achievement indirectly through increased absences, if feeling unsafe 
increases the likelihood that a student stays home out of fear. I test both the direct and indirect 
effects of reported feelings of safety on test scores to identify whether the impact of feeling 
unsafe operates solely through increased absences, or whether feeling unsafe exerts a unique 
effect on achievement in addition to any effect on absences. 
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Figure 1 : Theoretical Relationship between Safety, Absences, and Test Scores 
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METHOD 

Data 

Student surveys are the primary source of information for researchers and policymakers 
about violence and safety in the nation’s schools (Skiba et al. 2006). In 2007, the New York City 
Department of Education implemented a school environment survey for all students in sixth 
grade and above. The annual survey asks a series of questions about student engagement, 
school climate, and safety. This analysis is based upon student-level survey data for the 2006-7 
through 2009-10 school years. Over 80 percent of the middle school student in the district 
responded to the survey in more than 700 public schools and 10,000 homerooms. The survey 
data is matched to individual administrative education records from the Department of 
Education, providing a rich set of covariate and outcome measures, including the number of 
absences per year and standardized test scores. 

Response Rates and Survey Reliability 

The quality of research based on anonymous survey data rests on the validity and 
reliability of the reported information. Due to the high coverage of the student population in 
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grades six and above, the results of this study are generalizable to the population of surveyed 
students in New York City. While response rates vary across schools (Figure 2), the overall 
response rate is very high (above 80 percent). Descriptive analyses indicate that there are 
differences between students who take the survey and those who do not. Although these 
respondents and non-respondents are comparable in many ways, non-respondents have lower 
test scores on average than respondents. 1 These differences are potentially problematic if the 
students who do not respond to the survey have systematically different feelings of safety than 
respondents. Non-response could bias the results in either direction depending on whether non- 
respondents feel more or less safe than respondents. 


Figure 2: Distribution of Response Rates across Schools (2010) 
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I conduct two tests of the construct validity of the safety measures used in this study. 
First, for respondents, there is a strong correlation between reporting frequently staying home 
because of feeling unsafe at school and actual school absences recorded by the school, 
indicating that safety has important implications for attendance rates. 2 Students who feel the 
least safe may have more absences which make them more likely miss school when the survey 


1 Percentages of students who are female, enrolled in ESL, native born, and receiving free or reduced price lunch are comparable 
between respondents and non-respondents. Black and Hispanic students make up larger percentages of the non-respondent group 
than the respondent group. (Contact author for table). 

2 Students who stay home because they feel unsafe have 2.6 more absences on average, than students who do not. 
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or the standardized test is administered. As a result, the findings presented here may be 
underestimates of the true impact of feeling unsafe at school on test scores. 

As a second test, student perceptions of social disorder are compared to school-level 
administrative measures of school violence reported on an annual basis through the New York 
State Violent and Disruptive Incident Reporting (VADIR) system. 3 Figure 3 shows that student- 
reported violence and disorder varies in the expected direction with the level of school violence 
reported through the VADIR, indicating that students in the most violent schools report higher 
levels of disorder. 


Figure 3: Perceived Disorder by Level of School Violence 



Issues of reporting accuracy, particularly under-reporting or over-reporting on sensitive 
topics, could be a concern for research about feelings of safety. For instance, social pressures 
may make it likely for middle school students, particularly boys, to under-report feeing unsafe at 


3 Schools are categorized by quartiles based on the number of incidents that occur in a given year: "Low” = 25th percentile and 
below; “Mid” = between 25th and 50th percentiles; “Mod” = between 50th and 75th percentiles; “High” = 75th percentile and above. 
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school. In this case, significant effects of feeling unsafe on academic outcomes would be 
conservative estimates. Even with the likelihood of under-reporting, the survey data show that 
among middle school students in New York City, boys are more likely to report that they feel 
unsafe than girls. 

Sample 

The sample is restricted to students in the 6, 7, and 8 grades for three primary reasons. 
First, survey response rates are highest for students in these grades (compared to high school 
grades), ensuring better coverage of the population of middle school students. Second, feelings 
of safety in the classroom vary by grade level: the share of student that feels unsafe peaks in 
the 7th and 8th grades and declines as students enter high school. Therefore, safety may affect 
test scores for the largest number of students in the middle grades. Third, high school students 
do not take the state standardized exams and instead take a series of subject-area exams 
throughout their high school tenure, making it difficult to model test score changes between 
middle and high school. To ensure sufficient variation within schools and classrooms, schools 
with fewer than 10 respondents, and classrooms with fewer than 4 respondents are omitted 
from the analysis. The final sample includes survey responses for more than 340,000 individual 
students, over multiple years. 

Measures 

The main focus of this paper is safety in the classroom, given the direct link to academic 
performance. Responses to the four-response scaled survey item “I am safe in my classes” are 
re-coded as binary, taking a value of one if the student “disagrees” or “strongly disagrees” with 
the statement. Whether a student feels safe or unsafe is more salient for this analysis than the 
marginal difference between students who “disagree” or “strongly disagree” with the statement, 
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which would require strong assumptions about individual interpretations of these categories.4 
Using the same approach, I also construct measures of reported feelings of safety in the 
hallways, bathrooms, and locker rooms, and safety outside the school on school grounds. The 
final safety measure captures the frequency with which a student chooses to stay home 
because he or she feels unsafe at school. Students respond to the statement “I stay home 
because I don’t feel safe at school” with the frequency response options “never”, “some of the 
time,” “most of the time,” and “all of the time.” This measure is coded as a binary variable taking 
the value of one if the student stays home out of fear “most” or “all” of the time. 

The primary outcome measure, academic achievement, is parameterized by scores on 
an annual state math exam, standardized as z-scores by year and grade. To ensure temporal 
precedence, impacts are only estimated for math exam scores, because the math exams were 
administered after the student survey. During most of the study period, the English exams were 
administered prior to the school survey. The measure of school absences is the natural log of 
the number of full days absent in the past year. Some models also include individual student 
characteristics, including special education status, free or reduced price lunch status, whether 
the student speaks a language at home other than English, gender, race, and ethnicity. These 
indicator variables take the value of one if the characteristic is present for the student. 

Baseline Model 

The impact of feeling unsafe on academic achievement is estimated using a series of 
regression models. The first specification (equation 1) presents the baseline model of the 
relationship between feeling unsafe in the classroom ( UnsafeClass it ) and math test scores 
( TestScore it ), 


4 Models disaggregating the “disagree” and “strongly disagree” responses are presented as a robustness check. 
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(1) TestScore it = /? 0 + UnsafeClass it + ygrade * year t + %, 


where grade*year t is a set of dummy variables controlling for annual time trends at the 
grade level. However, individual student characteristics may explain both feeling unsafe in the 
classroom and achievement. The second specification (equation 2) includes a vector of 
observed student characteristics {IndChars, t ), 

(2) TestScore it = /3 0 + l3 1 UnsafeClasSi t + 8IndChars it + y grade * year t + s it , 

that includes special education status, free or reduced price lunch status, language 
spoken at home other than English, gender, and race and ethnicity. These models also include 
annual school enrollment to control for the effect of attending a larger school. 

Strengthening the Baseline Model 

There are several methodological challenges to isolating the impact of feeling unsafe on 
academic achievement. A main concern is omitted variable bias which could occur because 
school or classroom characteristics, such as the school environment or a particular teacher, 
affect both feelings of safety and academic achievement. If important variables are omitted from 
the model, changes in academic outcomes may be inaccurately attributed to students’ feelings 
of safety. As shown earlier, a larger share of students who attend more violent schools report 
feeling unsafe, compared to students who attend less violent schools. To strengthen the model, 
school fixed effects are added to control for characteristics of the school environment, such as 
violence and disorder, which may affect feelings of safety. 

However, within-school tracking into different classrooms may result in exposure to 
different classroom environments, even within the same school. Therefore, the next model 
includes homeroom fixed effects, controlling for unobserved, time-invariant characteristics of 
homerooms that likely affect safety and achievement. 
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Still, unobserved individual factors may be explaining student feelings of safety and 
academic achievement among students in the same homerooms. To strengthen the individual 
controls in the model, two approaches are taken. First, a value-added model is estimated in 
which the individual student’s test score from the previous year is added to the homeroom fixed 
effects model. In a value-added model, an individual’s achievement in a given year is a function 
of all previous years of schooling and experience. The value-added model estimates the effect 
of feeling unsafe in the classroom on the gain in test scores over the prior year alone, removing 
differences between students that have accumulated over years of schooling. 5 

Although it is an improvement over the previous model, the value-added model does not 
control for all time-invariant characteristics of an individual student that may be related to safety. 
Student fixed effects estimators are widely used in the literature about value-added models of 
education to achieve causal estimates of the impact of a policy change on student achievement 
(Gentile and Imberman 2011). Therefore, the second approach is to control for unobserved 
student characteristics that remain constant over time in a student fixed effects model. This 
model also includes controls for school-level annual time trends to capture the influence of 
shocks to the entire school - such as a new principal, or adoption of a new academic policy - 
that might affect both student safety and test scores. 

Prior research has documented differences in feelings of safety at school between black 
and white students, Hispanic and white students, and boys and girls (e.g., Alvarez and 
Bachman, 1997; Arum, 2003; Hong and Eamon 201 1 ; Schreck and Miller 2003). To investigate 
whether the impact of feeling unsafe in the classroom on academic outcomes differs by these 
characteristics, the student fixed effect model includes interactions between feeling unsafe in 
the classroom and race, ethnicity, and gender indicators. 

5 There is no clear standard in the literature about specification of value-added models. Most researchers use a cumulative model 
which estimates the impact of inputs on the level test score controlling for prior scores, or a gain score model which uses the change 
in test score from the previous year as the dependent variable (Gentile and Imberman 201 1 ; Harris and Sass 2006; Rockoff 2004; 
Rothstein 2009; Wiswall 2011). Another approach is to measure the contemporaneous effect of inputs on test scores using the 
student fixed effect alone to capture prior performance (Harris and Sass 2006; Wiswall 201 1 ). Value-added models are inconsistent 
when estimated using a random effects estimator, therefore a fixed effects estimator is used (Harris and Sass 2006). 
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To test the direct and indirect effects of feeling unsafe in the classroom, I add a measure 
of whether a student reports staying home from school because of feeling unsafe and a 
measure of the number of absences that occurred in the given year to test the moderating effect 
of absenteeism. 

Validity Tests 

Despite the strategies described above, there may remain a concern that systematic, 
time-varying individual or classroom characteristics not included in these models are driving 
both feelings of safety and academic achievement. Potential sources of omitted variable bias 
include changes in the home lives of individual students or general school or classroom 
“disorder” that might both cause students to feel unsafe and affect test scores independently. To 
isolate the impact of feeling unsafe on achievement from the other ways in which school 
disorder may affect test scores, I test the impact of alternative measures of safety at school that 
are less directly related to academic performance. If these measures of safety are also 
significant predictors of achievement, it is more likely that an omitted variable is causing 
students to feel unsafe across all contexts and their academic achievement to decline. 

However, if these measures are not significant predictors of test scores, any omitted variable 
that would threaten the validity of the causal inference would need to operate solely through 
feelings of safety in the classroom. 

In addition to omitted variable bias, another threat to the validity of the causal inference 
is reverse causality - an inability to determine the direction of causality between feeling unsafe 
at school and having poor academic performance. This can result in correlation between the 
independent variables in the model (i.e. feeling unsafe) and the error term, violating a condition 
of unbiased OLS estimation. The econometric models presented thus far have been based on 
prior research that finds exposure to violence negatively affects the academic achievement of 
students and their peers (Carrell and Hoekstra 2010; Sharkey 2010). If students exposed to 
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violence become fearful, they may misbehave in class and disturb their own learning and that of 
their peers, or they may stop coming to school altogether. If this is the case, one would expect 
the relationship to operate from feelings of safety to academic outcomes. However, one could 
tell a different, but plausible, story. Students who are falling behind in school may fear 
disappointing their teachers or being embarrassed in front of classmates, and may feel that the 
classroom is not a safe or comfortable place. In this case, poor academic performance may 
drive feelings of safety at school. On the other end of the spectrum, strong academic 
performance may be associated with feeling unsafe at school if high-performing students are 
targeted for bullying. To address the simultaneity concern, the analysis is restricted to 
standardized tests that are given after the survey is administered, and I conduct a falsification 
test of the impact of feeling unsafe in future years on current test scores. 

After isolating the relationship between classroom safety and academic performance 
and determining the direction of causality, I explore variation in the impact estimate across 
schools with different levels of school violence. 

Results 

Students who report feeling unsafe in the classroom experience a consistent, negative 
effect on test scores. The finding is robust to school, homeroom, and student fixed effects 
models. Robustness and validity checks support this central finding. 

Descriptive Statistics 

Fifteen percent of all middle school students report feeling unsafe in the classroom. 6 
Students who report feeling unsafe have different average characteristics than students who 
report feeling safe in the classroom (Table 1). Although the majority of students qualifies for free 
or reduced price lunch, a proxy for poverty, students who report feeling unsafe are more likely to 

6 Share of students by response to statement “I feel safe in the classroom”: Strongly agree (37%), Agree (43%), Disagree (10%), 
Strongly Disagree (5%), No response (6%). 
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be poor. 7 Males make up a larger share of students who report feeling unsafe in the classroom 
than females, and a larger share of black students report feeling unsafe than white, Asian, and 
Hispanic students. Students who feel unsafe in the classroom are more likely to qualify for 
special education services, compared to students who feel safe. 

Table 1 : Mean Characteristics of New York City public middle school students, 

by question response 


"I am safe in my classes. " 


Mean Student and School 
Characteristics 

Total 

Safe 

Unsafe 

No Response 

Observations 

658,122 

527,122 

93,418 

37,576 

Free/Reduced Lunch 

0.66 

0.65 

0.69 

0.70 

Female 

0.51 

0.52 

0.47 

0.44 

Home Language not English 

0.56 

0.56 

0.53 

0.59 

Special Education 

0.12 

0.11 

0.13 

0.17 

Black 

0.30 

0.28 

0.37 

0.39 

White 

0.15 

0.16 

0.11 

0.11 

Asian 

0.16 

0.17 

0.14 

0.10 

Hispanic 

0.39 

0.39 

0.38 

0.40 

Days Absent 

11.8 

11.4 

13.3 

14.1 

Took Math test (%) 

0.97 

0.97 

0.97 

0.96 

ELA Z score 

0.052 

0.111 

-0.159 

-0.267 

Math Z score 

0.073 

0.143 

-0.167 

-0.316 

Total Enrollment ('000s) 

692 

697 

682 

647 

Peers Same Race (%) 

0.52 

0.51 

0.53 

0.53 

Peer Social Disorder (%) 

0.09 

0.08 

0.11 

0.10 

Unsafe in Halls (%) 

0.28 

0.19 

0.80 

0.39 

Unsafe Outside (%) 

0.34 

0.26 

0.77 

0.43 

Stays Home Most or All of the time (%) 

0.05 

0.03 

0.15 

0.09 


Students who report feeling unsafe in the classroom have higher mean absences and 
lower scores on the math and English language arts standardized tests. The share of students 
who take standardized tests is high across all response categories (97 percent), tempering any 
concerns about systematic differences in test taking. While the average student across both 
safety responses is in a school that is majority same race or ethnicity, students who report 

7 Unsafe includes the responses “disagree” or “strongly disagree” to the statement “I feel safe in the classroom.” 
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feeling unsafe in the classroom go to schools where a larger share of peers report that social 
disorder -bullying, fighting, and gang activity - is a problem in the school. 

Reporting feeling unsafe in the classroom is correlated with reported feelings of safety in 
other areas of the school. Approximately 80 percent of students who feel unsafe in the 
classroom also feel unsafe in the hallways, bathrooms, and locker rooms, and outside the 
school on school grounds, compared to 19 percent of students who feel safe in the classroom 
but unsafe in the halls, and 26 percent of students who feel safe in the classroom but unsafe 
outside the school. Most notably, 15 percent of students who feel unsafe in the classroom also 
report that they stay home most or all of the time because they feel unsafe at school. Only 3 
percent of students who feel safe in the classroom report staying home out of fear at similar 
levels. 

Overall, students who respond to the survey but do not answer the safety questions 
appear to be similar to the least safe students. Compared to students who responded, larger 
shares of non-respondents are poor (70 percent), speak a language at home other than English 
(59 percent), and are enrolled in special education (17 percent). A larger share of the students 
who did not respond is black (39 percent). The mean number of absences and the mean 
reading and math scores for students who did not answer the safety questions are on par with 
or lower than students who report feeling the least safe. These statistics indicate that students 
who did not answer the classroom safety question are most similar to those students who feel 
the least safe at school, therefore the estimates of the impact of feeling unsafe on test scores 
may be underestimates. 

How safe students feel changes over time (Table 2). Of the students who reported 
feeling the least safe in the classroom in a prior year (strongly disagree), 38 percent continue to 
feel unsafe in the following year, while 57 percent report feeling safe in the classroom in the 
following year (and 5 percent are missing responses to the safety question). Of the students 
who feel the least safe in the current year, 25 percent changed their response from “strongly 


IESP Working Paper #02-13 17 


agree” in the previous year, 37 percent changed their response from “agree” from the previous 
year, 20 percent changed their response from “disagree” in the previous year, and 18 percent 
did not change their response. There does not seem to be a pattern of non-response linked to 
prior year response - if anything, of the students missing responses to the survey in the current 
year, over 80 percent reported feeling safe in the classroom in the prior year. 

Table 2. Within-Student Changes in Reported Feelings of Safety in the Classroom 


Statement: 

classes.” 

“1 am safe in my 



Current Year 



Strong 

Agree 

Agree 

Disagree 

Strong 

Disagree 

Missing 


Strong Agree 

0.51 

0.37 

0.05 

0.03 

0.04 

Previous 

Agree 

0.27 

0.56 

0.10 

0.04 

0.04 

Year 

Disagree 

0.18 

0.48 

0.20 

0.09 

0.05 


Strong Disagree 

0.21 

0.36 

0.19 

0.19 

0.05 


Regression Results 

The baseline specifications presented in Table 3 show consistent evidence that feeling 
unsafe in the classroom decreases test scores. This finding is robust to the addition of individual 
covariates, school and homeroom fixed effects, and inclusion of the prior year test score. The 
raw correlation between reporting feeling unsafe and test scores is a 0.32 standard deviation 
decrease in scores. The effect size is reduced significantly with the addition of individual 
covariates (0.23), school fixed effects (0.13), and homeroom fixed effects (0.09). The value- 
added specification (column 5) shows that reporting feeling unsafe in the classroom decreases 
math test scores by 0.06 standard deviations, controlling for prior test scores, homeroom 
effects, and grade level time trends. This effect size is larger than the independent effect of 
being poor on test scores (0.02). In column 6, the effects of “strongly disagreeing” and 
“disagreeing” are estimated separately, with the expected pattern of a stronger expression of 
feeling unsafe related to a larger decrease in test scores (0.07). 
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Table 3: Baseline Relationship between Feeling Unsafe and Math Z Scores 



(1) 

(2) 

(3) 

(4) 

(5) 

(6) 





Homeroom 

Value- 


MATH Z SCORE 

Raw 

Covariates 

School FE 

FE 

Added 

Categories 

Unsafe in Class 

-0.316*** 

-0.226*** 

-0.134*** 

-0.0894*** 

-0.0557*** 



(0.0178) 

(0.0104) 

(0.00289) 

(0.00270) 

(0.00210) 


Safe in Class: 
Disagree 






-0.0374*** 

(0.00242) 

Safe in Class: 
Strongly Disagree 






-0.0696*** 

(0.00345) 

White 


0.615*** 

0.329*** 

0.226*** 

0.0873*** 

0.0886*** 



(0.0336) 

(0.00442) 

(0.00415) 

(0.00328) 

(0.00319) 

Hispanic 


0.0714*** 

0.0482*** 

0.0397*** 

0.00289 

0.00535* 



(0.0186) 

(0.00319) 

(0.00300) 

(0.00231) 

(0.00225) 

Asian 


0.949*** 

0.699*** 

0.539*** 

0.251*** 

0.253*** 



(0.0405) 

(0.00456) 

(0.00423) 

(0.00336) 

(0.00328) 

Female 


0.00578 

-0.00419* 

-0.0264*** 

0.0139*** 

0.0152*** 



(0.00421) 

(0.00211) 

(0.00195) 

(0.00155) 

(0.00150) 

Home Lang, not 
English 


0.0207 

0.0454*** 

0.0816*** 

0.0549*** 

0.0533*** 



(0.0165) 

(0.00288) 

(0.00274) 

(0.00217) 

(0.00212) 

Free/Reduced Lunch 


-0.197*** 

-0.112*** 

-0.0450*** 

-0.0193*** 

-0.0190*** 



(0.0162) 

(0.00257) 

(0.00329) 

(0.00260) 

(0.00252) 

Special Education 


-0.663*** 

-0.620*** 

-0.491*** 

-0.218*** 

-0.220*** 



(0.0125) 

(0.00330) 

(0.00332) 

(0.00265) 

(0.00258) 

Enrollment (‘000s) 


0.0749* 

-0.0493*** 

-0.0288 

-0.0442** 

-0.0382** 



(0.0338) 

(0.0144) 

(0.0173) 

(0.0136) 

(0.0132) 

Math Z Score (t-1) 





0.593*** 

0.594*** 






(0.00139) 

(0.00135) 

Constant 

0.178*** 

0.00912 

0.0665*** 

0.00369 

-0.0329** 

-0.0466*** 


(0.0318) 

(0.0313) 

(0.0104) 

(0.0157) 

(0.0127) 

(0.0122) 

Observations 

586553 

586553 

586553 

586553 

586553 

620828 

R-squared 

0.014 

0.236 

0.339 

0.475 

0.671 

0.672 

Year*Grade FE 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

School FE 

No 

No 

Yes 

No 

No 

No 

Homeroom FE 

No 

No 

No 

Yes 

Yes 

Yes 


Clustered standard errors in parentheses 
*** p<0.01, ** p<0.05, * p<0.1 


Still, individual level omitted variables may explain the relationship between feeling 
unsafe and achievement. Controlling for time-invariant student characteristics further reduces 
the size of the effect of feeling unsafe (Table 4). The first student fixed effects model shows that 
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feeling unsafe in the classroom results in a 0.04 standard deviation decrease in test scores. The 
specification in column 2 includes annual school trends to control for school-wide changes that 
might affect test scores and safety (such as a new principal, or change in disciplinary or security 
policy). With these controls, on average, a student who reports feeling unsafe in the classroom 
experiences a 0.03 standard deviation decrease in math test scores. 8 


Table 4: Student Fixed Effect Models 



(1) 

(2) 

(3) 



School*Year 


MATH Z SCORE 

Student FE 

FE 

Categories 

Unsafe in Class 

-0.0353*** 

-0.0290*** 



(0.00529) 

(0.00482) 


Category of Unsafe in 
Class 



-0.0147*** 

(0.00240) 

Enrollment (‘000s) 

-0.0929** 

-0.0905 

-0.0792 


(0.0284) 

(0.0533) 

(0.0483) 

Grade 7 


-0.165*** 

-0.175*** 



(0.0315) 

(0.0320) 

Grade8 


-0.373*** 

-0.384*** 



(0.0591) 

(0.0601) 

Absences 



-0.00619*** 

(0.000294) 

Observations 

586553 

586553 

586510 

R-squared 

0.908 

0.914 

0.915 

Year*Grade FE 

Yes 

No 

No 

Student FE 

Yes 

Yes 

Yes 

SchooFYear FE 

No 

Yes 

Yes 


Although the descriptive statistics show that a greater share of black and Hispanic 
students feel unsafe in the classroom compared to white and Asian students, interaction models 
show no differences in the rate at which reporting feeling unsafe affects test scores by student 


8 The third specification estimates the effect of a change in response to a more “unsafe” category over time, regardless of whether it 
is a change from very safe to safe or from unsafe to very unsafe, and finds that overall, each decrease in reported feelings of safety 
results in a 0.02 standard deviation decrease in test scores. 
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race and ethnicity (contact author for table). One might assume that boys entering middle 
school are more likely to become involved in delinquent behavior, making them feel more fearful 
and also negatively influencing their academic achievement. On the contrary, the results show 
no differential effect of changes in feelings of safety in the classroom on test scores between 
boys and girls. 

Staying home from school because of fear is one mechanism through which feeling 
unsafe in the classroom may affect test scores. On average, students who indicate that they 
stay home because they feel unsafe at school should have a higher number of absences. As a 
validity check of this measure of safety, Table 5 presents the relationship in a regression 
framework. There is a strong association between staying at home because of feeling unsafe 
and the log of full day absences. The association persists with the inclusion of individual student 
characteristics, school and classroom fixed effects, and student fixed effects. Within the same 
classrooms, students who report staying home because they feel unsafe have 15 percent more 
absences, on average, than students who do not (specification 4). 

The results in Table 6 show that holding reported feelings of safety in the classroom 
constant, students who indicate that they stay home because they feel unsafe at school 
experience a larger decrease in test scores (an additional 0.03 standard deviation decrease). 9 
Each additional absence from school also decreases test scores. In fact, there is no 
independent effect of staying home because of feeling unsafe at school when an interaction 
term between staying home and absences is included in the model (column 3). The point 
estimate on classroom safety is largely unaffected by the inclusion of these additional 
measures, indicating that there is both a direct effect of feeling unsafe on academic 
achievement and an indirect effect through increased absences. 


9 The measure includes students who said that they “most” or “all” of the time stay home because of feeling unsafe. 
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Table 5: Relationship between Staying Home because of Feeling Unsafe 

and School Absences 


DV: log(Absences) 

(1) 

(2) 

( 3 ) 

( 4 ) 

( 5 ) 

(6) 





Homeroom 

Student 

Student FE and 

VARIABLES 

Raw 

Covariates 

School FE 

FE 

FE 

School Trend 

Stay Home b/c Feel 
Unsafe at School 

0.242*** 

0.203*** 

0.166*** 

0.145*** 

0.0485*** 

0.0401*** 


(0.00897) 

(0.00729) 

(0.00662) 

(0.00582) 

(0.00930) 

(0.0086) 

Black 


-0.0349 

-0.138*** 

-0.193*** 





(0.0284) 

(0.0198) 

(0.00606) 



Hispanic 


0.141*** 

0.0303* 

-0.0161*** 





(0.0262) 

(0.0171) 

(0.00534) 



Asian 


-0.536*** 

-0.531*** 

-0.503*** 





(0.0382) 

(0.0269) 

(0.00670) 



Female 


-0.0301*** 

-0.0225*** 

-0.0154*** 





(0.00420) 

(0.00373) 

(0.00261) 



Free/Reduced Lunch 


0.171*** 

0.133*** 

0.0684*** 





(0.0132) 

(0.00804) 

(0.00470) 



Home Lang, not 
English 


-0.236*** 

-0.222*** 

-0.228*** 





(0.0146) 

(0.00880) 

(0.00400) 



Special Education 


0.204*** 

0.184*** 

0.120*** 





(0.00833) 

(0.00624) 

(0.00448) 



Enrollment ('000s) 


-0.0245 

0.118*** 

0.0739** 

0.0183 

0.1296 



(0.0324) 

(0.0424) 

(0.0312) 

(0.0370) 

(0.0680) 

Constant 

2.067*** 

2.183*** 

2.181*** 

2.405*** 

1.901*** 



(0.0174) 

(0.0358) 

(0.0308) 

(0.0263) 

(0.0394) 


Observations 

542,765 

542,765 

542,765 

542,765 

542,765 

542,765 

R-squared 

0.003 

0.099 

0.162 

0.227 

0.868 

0.876 

Grade*Year FE 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

Covariates 

No 

Yes 

Yes 

Yes 

No 

No 

School FE 

No 

No 

Yes 

No 

No 

No 

Homeroom FE 

No 

No 

No 

Yes 

No 

No 

Student FE 

No 

No 

No 

No 

Yes 

Yes 

School*Year FE 

No 

No 

No 

No 

No 

Yes 


Robust standard errors in parentheses 
*** p<0.01, ** p<0.05, * p<0.1 
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Table 6: Mechanisms 


VARIABLES 

(1) 

Student FE 

(2) 

Mediators 

(3) 

Interaction 

Unsafe in Class 

-0.0294*** 

-0.0243*** 

-0.0242*** 

Stay Home b/c 

(0.00493) 

(0.00503) 

(0.00503) 

Unsafe 


-0.0329*** 

(0.00699) 

-0.0176 

(0.0103) 

Log(Absences) 

Stay 

Home*log(Absences) 


-0.00617*** 

(0.000301) 

-0.00608*** 

(0.000300) 

-0.00114* 

(0.000560) 

Enrollment ('000s) 

-0.103 

-0.0973 

-0.0980 


(0.0548) 

(0.0551) 

(0.0552) 

Observations 

575,286 

575,286 

575,286 

R-squared 

0.915 

0.915 

0.915 

Student FE 

Yes 

Yes 

Yes 

Grade FE 

Yes 

Yes 

Yes 

School*Year FE 

Yes 

Yes 

Yes 


Robust standard errors in parentheses 
*** p<0.01,** p<0.05, * p<0.1 


Validity Tests 

It is possible that unobserved, time-varying student characteristics may explain both 
feelings of safety and test scores, or that potential simultaneity prevents the identification of a 
causal relationship. Although there is no surefire way to alleviate this concern in the quasi- 
experimental context, several validity tests provide support that the estimates are causal. 

For the first validity test, I estimate a model including alternative measures of feelings of 
safety in school that are less closely related to academic performance. If feeling unsafe in all 
contexts affects test scores, it is more likely that an omitted variable that is affecting safety and 
achievement is present and causing bias in the results. For instance, the victimization of a 
family member through domestic violence may influence both feelings of safety and 
performance in school. Flowever, one would expect that this type of violence exposure would 
make students feel unsafe in all contexts, not just in the classroom. Table 7 provides the results 
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from variants of the student fixed effect model that include feeling unsafe in the hallways, 
bathrooms, and locker rooms at school, and feeling unsafe outside the school on school 
grounds. There is no relationship between these measures of safety and test scores when 
controlling for safety in the classroom, and the magnitude and significance of the effect of 
feeling unsafe in the classroom is unchanged. Any remaining problematic omitted variable 
affecting test scores must be related to feelings of safety in the classroom only. 


Table 7: Validity Test #1, Impact of Other Safety Measures on Math Z Scores 


VARIABLES 

(1) 

Unsafe in Halls 
DV: Math Z Score 

(2) 

Unsafe Outside 
DV: Math Z Score 

Unsafe in Class 

-0.0228*** 

-0.0224*** 


(0.00539) 

(0.00537) 

Unsafe in Halls 

-0.00293 

(0.00444) 


Unsafe Outside 


-0.00499 

(0.00392) 

Stay Home b/c Unsafe 

-0.0327*** 

-0.0325*** 


(0.00730) 

(0.00728) 

Absences 

-0.00618*** 

-0.00618*** 


(0.000301) 

(0.000301) 

Enrollment ('000s) 

-0.0794 

-0.0796 


(0.0520) 

(0.0520) 

Observations 

560355 

560355 

R-squared 

0.917 

0.917 

Student FE 

Yes 

Yes 

Grade FE 

Yes 

Yes 

SchooFYear FE 

Yes 

Yes 


Robust standard errors in parentheses 
*** p<0.01, ** p<0.05, * p<0.1 


Second, as a falsification check, I test whether feelings of safety in the following year 
predict test scores in the current year (Table 8). No relationship is found between future safety 
and current test scores, and the coefficient on the current year safety measure is unchanged. 
Future reported feelings of safety do not affect test scores in the previous year, suggesting that 
changes in reported feelings of safety are driving decreases in test scores, not the reverse. 
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Table 8: Validity Test #2, Falsification Test 


Falsification Test 
VARIABLES 

(1) 

Reference 
DV: Math Z Score 

(2) 

Falsification Test 
DV: Math Z Score 

Unsafe in Class 

-0.0253*** 

-0.0245** 


(0.00704) 

(0.00776) 

Unsafe in Class (t+1) 


0.00202 



(0.00777) 

Enrollment ('000s) 

-0.0562 

-0.0565 


(0.0745) 

(0.0745) 

Observations 

332138 

332138 

R-squared 

0.929 

0.929 

Student FE 

Yes 

Yes 

Grade FE 

Yes 

Yes 

SchooFYear FE 

Yes 

Yes 


Robust standard errors in parentheses 
*** p<0.01, ** p<0.05, * p<0.1 


The findings are also robust to estimation on a smaller, balanced panel of students who 
respond to the survey for all three years of middle school (Table 9). The coefficient on feeling 
unsafe in the classroom maintains significance and the point estimate from the balanced panel 
model is larger than the estimates achieved using the unbalanced panel. As a final robustness 
test, I estimate variations on the value-added model, including a model with a lagged test score 
but no student fixed effects (as employed by Gottfried 2010), a student fixed effect model with 
level scores and then with lagged scores, and a model of the change in test score as the 
dependent variable (Table 10). The point estimate and significance of the variable of interest is 
larger for the first specification, indicating that a value-added approach alone is no substitute for 
a student fixed effect estimator. Across the student fixed effect models, the estimates for feeling 
unsafe in the classroom are robust to estimation with just the level math score and no lagged 
score, and estimation on the change in math score. 
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Table 9: Robust Test #1, Balanced Panel 


Balanced Panel 

(1) 

(2) 


Unbalanced 

Balanced 

VARIABLES 

DV: Math Z Score 

DV: Math Z Score 

Unsafe in Class 

-0.0294*** 

-0.0323*** 


(0.00493) 

(0.00517) 

Enrollment ('000s) 

-0.103 

-0.0559 


(0.0548) 

(0.0593) 

Observations 

575286 

204731 

R-squared 

0.915 

0.862 

Student FE 

Yes 

Yes 

Grade FE 

Yes 

Yes 

SchooPYear FE 

Yes 

Yes 

Robust standard errors in parentheses 


*** p<0.01,** p<0.05, 

* p<0.1 


Table 10: Robust Test #2, Value-Added Specifications 


Value-Added 

VARIABLES 

(1) 

Lag Score 
DV: Math Z Score 

(2) 

Level 

DV: Math Z Score 

(3) 

Lag, FE 

DV: Math Z Score 

(4) 

Change 

DV: Math Z Score 

Unsafe in Class 

-0.0583*** 

-0.0290*** 

-0.0274*** 

-0.0348*** 


(0.00213) 

(0.00482) 

(0.00480) 

(0.00759) 

Math Z Score (t-1) 

0.626*** 


-0.269*** 



(0.00137) 


(0.00719) 


Enrollment (‘000s) 

-0.0405** 

-0.0905 

-0.0829 

-0.107 


(0.0138) 

(0.0533) 

(0.0548) 

(0.0776) 

Observations 

586553 

586553 

586553 

586553 

R-squared 

0.661 

0.914 

0.922 

0.505 

Classroom FE 

Yes 

No 

No 

No 

Student FE 

No 

Yes 

Yes 

Yes 

Grade*Year FE 

Yes 

No 

No 

No 

Grade FE 

No 

Yes 

Yes 

Yes 

SchooPYear FE 

No 

Yes 

Yes 

Yes 


Robust standard errors in parentheses 
*** p<0.01, ** p<0.05, * p<0.1 
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Variation in the Impact Estimates 

Finally, I investigate the variation in the impact estimates by levels of school violence to 
learn whether the impact of feeling unsafe on test scores is larger for students who are exposed 
to more school-based violence. Descriptively, it appears that the average student in a school 
with high violence feels less safe than the average student in a low violence school. Differences 
in the violent incident rate in the school may result in larger impacts of feeling unsafe in the 
classroom on academic outcomes. Table 1 1 presents the student fixed effect models stratified 
by quartiles of school violent and disruptive incident rates in 2007 (the baseline year of the 
survey). 10 Column 1 estimates the impact of feeling unsafe in the classroom on math scores for 
students who attend schools that had the lowest incident rates in the city in 2007, and column 4 
presents the impact estimate for students attending schools that had the highest incident rates. 
The models also control for the change in the incident rate between 2007 and 2010, and total 
enrollment in the school, as well as grade and year fixed effects. Results show that feeling 
unsafe in the classroom has no statistically significant effect on test scores for students in the 
lowest violence schools, but as students are exposed to greater in-school violence and 
disruption the impact estimates grow larger and become statistically significant. For students in 
schools with the highest violent and disruptive incident rates, the impact of feeling unsafe in the 
classroom is a -0.033 standard deviation decrease in math performance. 


10 School-based violent and disruptive event rates are based on the statewide Violent and Disruptive Incident Report (VADIR). 
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Table 1 1 : Impact of Feeling Unsafe in the Classroom on Math Z Scores, by Rate of in 
School-based Violent and Disruptive Incidents (2007) 


Impact Estimate 
Variation 

(1) 

Low Violence 
(Q1 : 

Rate < -30) 

(2) 

Moderate 
Violence (Q2: -30 
< Rate < 22) 

(3) 

Medium 
Violence (Q3: 
22 < Rate < 
121) 

(4) 

High Violence 
(Q4: Rate > 
121) 

Unsafe in Class 

-0.0307 

(0.0237) 

-0.0219* 

(0.0103) 

-0.0306** 

(0.0104) 

-0.0325*** 

(0.00791) 

Change in Violent 
Incident Rate 
(2007-2010) 

-0.0000198 

(0.000270) 

-0.000647 

(0.000823) 

-0.000207 

(0.00111) 

-0.000148 

(0.00202) 

Enrollment (‘000s) 

0.0395 

0.0429 

-0.0302 

-0.0948 


(0.142) 

(0.0828) 

(0.0869) 

(0.0711) 

Observations 

50447 

113070 

184815 

201415 

R-squared 

0.915 

0.914 

0.916 

0.919 

Student FE 

Yes 

Yes 

Yes 

Yes 

School*Year FE 

Yes 

Yes 

Yes 

Yes 

Grade FE 

Yes 

Yes 

Yes 

Yes 


Discussion 

This study provides the first large-scale estimate of the impact of feeling unsafe in the 
classroom on academic achievement, and indicates that the effect is non-trivial - a 0.03 
standard deviation decrease in math test scores in the most controlled, student fixed effects 
model. To put this estimate in context, the achievement gap between black students and white 
students widens by about 0.10 standard deviations a year (Fryer and Levitt 2004), and in New 
York City, the gap between black and white students is estimated to be 0.12 standard deviations 
for 8th grade students, and the Flispanic-white gap is 0.17 standard deviations (Stiefel et al. 
2006). 

Prior work has found that black and Hispanic students systematically report feeling less 
safe in the classroom than their white and Asian peers, even when they share the same schools 
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and homerooms (Author 2012). Based on the estimates from this analysis, differential feelings 
of safety in the classroom may contribute to between a quarter and a third of the gap in test 
scores between minority and white students. It is clear that ensuring that each student feels safe 
in the classroom is a critical first step in improving educational achievement and reducing racial 
and ethnic achievement gaps. These findings also suggest that one place to start would be in 
schools that consistently report the highest violent and disruptive incident rates, where students 
experience the largest decline in test scores as a result of feeling unsafe in the classroom. 

District, school, and classroom based strategies can contribute to improving student 
safety. For instance, at the district level in New York City, School Report Card grades account 
for school safety ratings and a “safe environment” is a category of evaluation during site visits 
for the Quality Reviews of all city schools. This research suggests that the student safety 
measures should carry larger weight in the Report Card grades and Quality Reviews, 
highlighting schools where students feel particularly unsafe. Further, attention should be paid to 
classroom safety and preventing absences due to safety issues. To identify successful school 
and classroom level interventions, more research is needed to investigate contextual factors 
that mediate and/or moderate the impact of feelings of safety on academic outcomes, and to 
highlight schools that promote safety among students who are exposed to dangerous or 
disorderly environments, allowing them to excel academically. 

This analysis provides a strong case for the identification of a causal relationship, 
although some limitations remain. The effect of feeling unsafe on academic outcomes may be 
particularly salient for acute events, such as exposure to violent crime that causes a direct, but a 
potentially short-lived effect on both safety and academic performance. Acute effects are difficult 
to detect in this model due to the annual observation of feelings of safety, therefore these 
results likely reflect the cumulative effect of feeling unsafe at school over time. Also, the school 
survey data provide detailed information about student perceptions of their environment that 
researchers generally do not have access to, but the questions about safety do not distinguish 
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between physical safety and other types of safety, such as intellectual or emotional safety. The 
effect of safety on academic achievement may differ by type of safety, but in this study I am 
unable to distinguish between these different types of safety in the student responses. 

Finally, most research focused on a single city or a sample of students suffers from 
limited external validity. In this case, high coverage of the student population makes it possible 
to generalize from the results to all New York City middle school students. The sheer size and 
diversity of the New York City public school system provides ample variation in race, ethnicity, 
immigrant status, and other student factors, making lessons from New York relevant for other 
large urban schools systems. Flowever, factors influencing safety may differ across other 
municipal contexts, and comparative work would benefit the field. 
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